0
1
00:00:00,300 --> 00:00:02,150
So hit our Jupyter notebook.
1

2
00:00:02,280 --> 00:00:09,360
Let's investigate how wrong and how useful our house price model is. And will commemorate this with a
2

3
00:00:09,360 --> 00:00:12,620
new section heading. I'm going to change this to Markdown here
3

4
00:00:12,810 --> 00:00:22,220
and put two hash tags there and write "Residuals & Residual Plots".
4

5
00:00:22,440 --> 00:00:29,310
And now what I'm going to do is I'm going to take some of our existing code and just copy it below.
5

6
00:00:29,610 --> 00:00:33,490
We're going to reuse some cells, but modify them a little bit.
6

7
00:00:33,840 --> 00:00:36,050
So I'm going to take this cell right here,
7

8
00:00:36,180 --> 00:00:44,880
copy it, scroll down and paste it in, and I'm going to add some comments here.
8

9
00:00:44,930 --> 00:00:57,020
I'm going to call this one "Modified model: transformed (using log prices) & simplified
9

10
00:00:59,290 --> 00:01:02,020
(dropping two features)".
10

11
00:01:02,100 --> 00:01:05,240
Now just because I put that in the comment doesn't mean we've actually done it.
11

12
00:01:05,290 --> 00:01:09,580
So I'm going to come here and on this line,
12

13
00:01:09,580 --> 00:01:19,200
we're not just going to drop the PRICE, we're going to put some square brackets around PRICE, then a comma and
13

14
00:01:19,200 --> 00:01:26,260
then we'll drop two other features that we had earlier -  Industry and AGE.
14

15
00:01:26,260 --> 00:01:32,320
Now let me come in here and delete this part of the code. I'm going to hit Shift+Enter,
15

16
00:01:32,590 --> 00:01:40,530
and now I'm going to add another comment here that reads "Using Statsmodel", so instead of our scikit-learn
16

17
00:01:40,740 --> 00:01:46,310
we're to continue using statsmodel here for analyzing our residuals. Next,
17

18
00:01:46,330 --> 00:01:53,440
I'm going to come up here, take these three lines of code, copy them and paste them in here.
18

19
00:01:56,850 --> 00:01:57,420
Okay,
19

20
00:01:57,420 --> 00:01:59,700
so now we're all set up for this lesson.
20

21
00:01:59,700 --> 00:02:03,220
Now we can start looking at our residuals.
21

22
00:02:03,300 --> 00:02:13,110
So I'm going to add a comment here. We said the residuals are equal to the difference between our observed
22

23
00:02:13,110 --> 00:02:17,670
values and our fitted values or our predictions.
23

24
00:02:17,700 --> 00:02:26,790
So the observed values y were y_train, and from y_train we can subtract the fitted values. We can find
24

25
00:02:26,790 --> 00:02:32,910
these under "results.fittedvalues".
25

26
00:02:32,920 --> 00:02:35,960
Let's take a look at what these residuals actually look like.
26

27
00:02:35,970 --> 00:02:42,870
So residuals and Shift+Enter will split and shuffle the data,
27

28
00:02:42,870 --> 00:02:47,100
train the model and then output our residuals.
28

29
00:02:47,220 --> 00:02:48,300
And here we go.
29

30
00:02:48,300 --> 00:02:54,090
These are all 404 residuals from our model.
30

31
00:02:54,090 --> 00:03:03,360
If you're wondering what type of object we're working with here, then we can see it is a pandas Series.
31

32
00:03:03,370 --> 00:03:09,810
That's quite nice because if it's a Pandas Series we can call a bunch of methods on it that the Series
32

33
00:03:09,810 --> 00:03:11,950
has, like describe,
33

34
00:03:12,600 --> 00:03:20,330
and if I do that, we can see that the mean of the residuals is very, very close to zero.
34

35
00:03:20,580 --> 00:03:22,820
But more on that later.
35

36
00:03:22,830 --> 00:03:31,330
So let me quickly comment this out, hit Shift+Enter and show you that you don't actually need to do
36

37
00:03:31,330 --> 00:03:37,250
this calculation manually, because our results object from our Statsmodel
37

38
00:03:37,390 --> 00:03:42,880
actually already has an attribute which allows you to fetch the residuals.
38

39
00:03:42,880 --> 00:03:49,400
So "results.resid" is the name of said attribute and hitting Shift+Enter,
39

40
00:03:49,530 --> 00:03:55,140
we get presented with exactly the same thing. All right,
40

41
00:03:55,160 --> 00:03:57,460
let me comment this code out again,
41

42
00:03:57,890 --> 00:03:59,310
scroll down a little bit,
42

43
00:03:59,350 --> 00:04:08,600
hit Shift+Enter and get to the interesting part, namely the part where we are graphing our stuff. So I'll
43

44
00:04:08,600 --> 00:04:20,460
add another comment here that reads "Graph of Actual vs. Predicted Prices". Now,
44

45
00:04:20,470 --> 00:04:26,090
before I write this code, let me explain the rationale of what I'm about to do here.
45

46
00:04:26,380 --> 00:04:32,090
I want to create a chart that visualizes how good our predictions are.
46

47
00:04:32,110 --> 00:04:39,700
I want to see how our predictions actually stack up to the actual property prices, because seeing this
47

48
00:04:39,700 --> 00:04:47,760
picture will put the r-squared from the regression that we calculated into context. If you remember from
48

49
00:04:47,760 --> 00:04:54,600
our earlier lessons, that r-squared was kind of a nice one number summary for our regression, but it's
49

50
00:04:54,600 --> 00:04:57,180
definitely not the only thing that we should be looking at.
50

51
00:04:57,250 --> 00:05:04,830
Now let me ask you this, on this chart of the actual versus the predicted prices, what would this chart
51

52
00:05:04,830 --> 00:05:09,450
look like if our model was perfectly accurate?
52

53
00:05:09,510 --> 00:05:16,360
If it had like perfect predictive capabilities, what would you expect to see? In that case,
53

54
00:05:16,400 --> 00:05:22,470
you'd expect that the predicted prices are kind of a really, really close match to the actual ones.
54

55
00:05:22,580 --> 00:05:28,790
In fact you'd get all of these things lining up on a straight line, right?
55

56
00:05:28,850 --> 00:05:36,180
You should have a very, very strong correlation between the model's predictions and the actual results.
56

57
00:05:36,230 --> 00:05:44,420
Let's calculate this correlation and store it in a variable. So I'm going to say "corr = y_
57

58
00:05:44,690 --> 00:05:54,910
train.corr(results.fittedvalues)".
58

59
00:05:55,180 --> 00:06:02,770
This here calculates the correlation method on our y_train object and we provide, as an argument,
59

60
00:06:03,310 --> 00:06:07,350
the fitted values from our regression.
60

61
00:06:07,570 --> 00:06:15,280
Let's output this value here, so I'm going to say corr and hit Shift+Enter. The correlation that we see printed out here
61

62
00:06:15,440 --> 00:06:19,140
is a 0.89 approximately.
62

63
00:06:19,150 --> 00:06:21,160
That's, that's quite high actually.
63

64
00:06:21,280 --> 00:06:23,010
Which is pretty good.
64

65
00:06:23,020 --> 00:06:30,850
I think one thing I'd like to do though is round this value here, so "round()" and then at the
65

66
00:06:30,850 --> 00:06:38,500
end here a comma and 2, so I'm going to round it to two decimal places close the parentheses and hit Shift+Enter
66

67
00:06:38,830 --> 00:06:39,540
again.
67

68
00:06:39,640 --> 00:06:41,460
Nice.
68

69
00:06:41,470 --> 00:06:47,860
Now let's create a scatter plot using matplotlib which we've stored in our plt variable.
69

70
00:06:48,350 --> 00:06:53,030
So I'm going to say "plt.scatter()"; for the x axis,
70

71
00:06:53,080 --> 00:06:59,950
we're gonna use our y_train, our actual values. For the y axis,
71

72
00:07:00,010 --> 00:07:01,690
we're gonna use our fitted values,
72

73
00:07:01,690 --> 00:07:06,310
so this was "results.fittedvalues".
73

74
00:07:06,310 --> 00:07:13,150
Now I'm also going to add a color and a transparency to this scatter plot. For the color,
74

75
00:07:13,150 --> 00:07:14,570
"c = ",
75

76
00:07:14,800 --> 00:07:23,770
I'm going to go for maybe navy blue, so 'navy' in single quotes, and for the alpha, the transparency,
76

77
00:07:23,860 --> 00:07:28,180
I'm gonna go for 0.6.
77

78
00:07:28,270 --> 00:07:33,460
Now let me write "plt.show()" and take a look at what this looks like.
78

79
00:07:33,460 --> 00:07:35,510
Here we go.
79

80
00:07:35,710 --> 00:07:39,120
Now, it's a little hard to see what's going on here.
80

81
00:07:39,130 --> 00:07:47,440
I think we need some labels on our axes and we should also plot a line on this graph that represents
81

82
00:07:47,440 --> 00:07:48,690
the perfect fit,
82

83
00:07:48,760 --> 00:07:53,410
so we have a reference point for this scatter plot.
83

84
00:07:53,410 --> 00:07:56,680
Let me hide this plot by double clicking in here
84

85
00:07:57,470 --> 00:08:03,710
and updating our code now. Now in adding our axis labels,
85

86
00:08:03,710 --> 00:08:12,850
I'm going to show you how you can include scientific notation or LaTeX notation on the labels themselves.
86

87
00:08:12,940 --> 00:08:19,250
LaTeX notation is something that we've used in our Markdown cell but so far we haven't really integrated
87

88
00:08:19,430 --> 00:08:20,450
into our charts.
88

89
00:08:20,510 --> 00:08:21,810
So let's do that now.
89

90
00:08:22,030 --> 00:08:29,990
I'm going to have "plt.xlabel('Actual log prices'),
90

91
00:08:30,440 --> 00:08:32,630
and here's the LaTeX notation:
91

92
00:08:32,630 --> 00:08:34,280
"$$",
92

93
00:08:34,460 --> 00:08:37,450
and then between the dollar signs I'm going to have 'y'
93

94
00:08:38,270 --> 00:08:41,310
and then I'm going to have like an underscore and then an "i",
94

95
00:08:42,370 --> 00:08:51,470
meaning this "i" will be shown as a subscript and anything between this opening and closing tag will be
95

96
00:08:51,470 --> 00:08:53,270
treated as LaTeX
96

97
00:08:53,390 --> 00:09:03,660
notation. Let me add a font size as well, so ", fontsize = 14".
97

98
00:09:03,770 --> 00:09:05,650
Let's see what this looks like with Shift+
98

99
00:09:05,660 --> 00:09:09,090
Enter.
99

100
00:09:09,100 --> 00:09:16,840
Here we go, as promised, we see that y has a subscript. Let's add the label for the y axis. "plt.
100

101
00:09:16,870 --> 00:09:18,550
ylabel()"
101

102
00:09:19,480 --> 00:09:20,780
And try this one more time,
102

103
00:09:20,800 --> 00:09:31,690
say "Predicted log prices" and for the LaTeX notation, I'm going to say y hat underscore y and I
103

104
00:09:31,690 --> 00:09:42,970
can do this by having that slash, "hat" for the LaTeX tag and then again I'll have "y _i"
104

105
00:09:42,980 --> 00:09:48,850
also "fontsize = 14".
105

106
00:09:48,960 --> 00:09:57,330
Take a look. Here we can see that the hat tag in the LaTeX notation creates this look carry symbol
106

107
00:09:57,900 --> 00:10:00,930
above our y here.
107

108
00:10:01,140 --> 00:10:02,160
Nice.
108

109
00:10:02,160 --> 00:10:04,550
Now let's add a title to this chart.
109

110
00:10:04,890 --> 00:10:14,340
And in the title we'll use both LaTeX notation and we'll use an f-string to show our correlation as
110

111
00:10:14,340 --> 00:10:15,100
well.
111

112
00:10:15,180 --> 00:10:28,920
So "plt.title(f'Actual vs Predicted log prices: 
112

113
00:10:29,090 --> 00:10:30,330
$$'",
113

114
00:10:30,690 --> 00:10:44,340
and between the dollar signs, "y _i$ vs $\hat y_i$",
114

115
00:10:44,350 --> 00:10:52,720
and then outside the dollar sign "(Corr {corr}), which is the name
115

116
00:10:53,140 --> 00:10:59,030
of our variable here. And at the end of course we've got closing curly braces,
116

117
00:10:59,170 --> 00:11:08,840
the closing parentheses for this one, single quotes finish it all off and maybe a fontsize of say,
117

118
00:11:08,910 --> 00:11:10,230
17.
118

119
00:11:10,300 --> 00:11:12,640
Let's take a look.
119

120
00:11:12,850 --> 00:11:15,220
Fatal Exception.
120

121
00:11:15,220 --> 00:11:17,290
What was the problem?
121

122
00:11:17,290 --> 00:11:21,470
A typo in the LaTeX tag.
122

123
00:11:21,580 --> 00:11:25,360
It should read "\hat".
123

124
00:11:25,390 --> 00:11:26,890
There we go.
124

125
00:11:26,890 --> 00:11:30,580
So I think this chast is already starting to look pretty good.
125

126
00:11:30,580 --> 00:11:37,510
The only thing missing is that line that represents perfect predictions and we can add that up here
126

127
00:11:37,540 --> 00:11:44,900
with "plt.plot()", plotting a single line and that line for the x and the y will be the same.
127

128
00:11:45,130 --> 00:11:53,890
So in this case it would be "y_train, y_train" and I'm gonna give it a different color as well, so
128

129
00:11:53,890 --> 00:12:02,160
color is equal to say "cyan". That'll make it stand out quite nicely, so check it out.
129

130
00:12:02,200 --> 00:12:03,330
Here we go.
130

131
00:12:03,550 --> 00:12:06,960
This cyan line would represent a perfect prediction.
131

132
00:12:07,030 --> 00:12:13,930
So it goes through all these grid points here, 2 and 2, 2.5 and 2.5,  3.5 and 3.5
132

133
00:12:13,930 --> 00:12:15,600
right?
133

134
00:12:15,680 --> 00:12:24,160
And this allows us to see how our predictions stack up to the actual values very, very clearly.
134

135
00:12:24,160 --> 00:12:29,530
The bigger the distance from the cyan line the bigger the residual.
135

136
00:12:29,530 --> 00:12:35,500
In other words, in this part of the chart our model is doing really, really well.
136

137
00:12:35,500 --> 00:12:40,850
But for these data points here we have much larger residuals.
137

138
00:12:40,990 --> 00:12:45,130
Now what if looking at log prices isn't that helpful?
138

139
00:12:45,130 --> 00:12:53,070
What if you wanted to look at the actual prices, the actual dollar values? Because that's much more intuitive.
139

140
00:12:53,080 --> 00:12:55,840
All we have to do is reverse our transformation, right?
140

141
00:12:56,470 --> 00:12:59,650
So let's create a second chart below this one.
141

142
00:12:59,800 --> 00:13:01,910
I'll take this code here.
142

143
00:13:01,960 --> 00:13:04,530
Copy it, paste it below.
143

144
00:13:05,410 --> 00:13:15,100
And then all I have to do to reverse the transformation is raise our data points to the power of s so
144

145
00:13:15,220 --> 00:13:17,730
"np.e**"
145

146
00:13:17,740 --> 00:13:23,040
which raises it to the power and we'll do the same thing here "np.e",
146

147
00:13:24,030 --> 00:13:28,320
raise it to the power of our residuals.
147

148
00:13:28,320 --> 00:13:30,780
And of course we have to do the same thing here, "np.e
148

149
00:13:30,810 --> 00:13:43,470
**", "np.e**" and then "Actual prices" in the label, "Predicted prices" in the label
149

150
00:13:43,590 --> 00:13:49,740
and then for the title we'll also do that. To differentiate the two charts visually,
150

151
00:13:49,830 --> 00:13:54,010
I'm just gonna go with straightforward "blue" for this one, so our scatter plot here
151

152
00:13:54,150 --> 00:14:04,930
will just be blue. Shift+Enter updates what we see, and now our axes are transformed and we see our dollar
152

153
00:14:04,930 --> 00:14:09,460
prices in thousands. We can even add that to the label.
153

154
00:14:09,670 --> 00:14:27,470
So "Actual prices 000s". And here "Predicted prices 000s". Shift+Enter, and now our labels are updated.
154

155
00:14:27,820 --> 00:14:33,400
Now that we've got our charts pretty much where we want them to be, we can start to interpret what it
155

156
00:14:33,400 --> 00:14:34,360
is that we're looking at.
156

157
00:14:35,550 --> 00:14:43,240
How do you think our predictions are stacking up to the actual values looking at these charts?
157

158
00:14:43,260 --> 00:14:45,100
I think we're doing actually rather well.
158

159
00:14:45,450 --> 00:14:49,890
And what we're seeing here pretty much ties out with the expectations that we had
159

160
00:14:49,890 --> 00:14:52,510
looking at the high r-squared.
160

161
00:14:52,710 --> 00:14:58,520
A lot of the predictions in fact are very close to this cyan line here.
161

162
00:14:58,520 --> 00:15:03,440
Mind you, this cyan is not the best fit line for the scatter plot.
162

163
00:15:03,440 --> 00:15:08,780
The line here shows where a points would have to be if our predictions were perfect and our residuals
163

164
00:15:08,810 --> 00:15:10,660
would be equal to zero.
164

165
00:15:10,760 --> 00:15:15,540
The cyan line represents where y hat would be equal to y.
165

166
00:15:15,860 --> 00:15:22,270
But looking at this chart I do see something that kind of concerns me a little bit.
166

167
00:15:22,280 --> 00:15:25,900
Take a look at this top right corner here.
167

168
00:15:25,940 --> 00:15:33,300
There's definitely something kind of strange happening with the properties at the very top price bracket.
168

169
00:15:33,380 --> 00:15:40,460
These are the properties at the fifty thousand dollar mark. Here we can see that our residuals are actually
169

170
00:15:40,460 --> 00:15:45,850
fairly large and our model is performing quite poorly.
170

171
00:15:45,950 --> 00:15:53,210
If you remember we actually visualized all our target values in our data exploration stage and then
171

172
00:15:53,210 --> 00:16:00,230
we already spotted that there were quite a few properties at the top end of the price bracket and that,
172

173
00:16:00,230 --> 00:16:07,460
given this histogram, these kind of struck us as outliers and we suspected that maybe these might cause
173

174
00:16:07,460 --> 00:16:10,080
an issue down the line.
174

175
00:16:10,160 --> 00:16:15,140
What I'm saying is is that - yes, there's room for improvement in our model, obviously.
175

176
00:16:15,350 --> 00:16:20,440
but overall I think I'm pretty happy with what we've got so far.
176

177
00:16:20,690 --> 00:16:28,220
After all, we're dealing with a real data set so it's nice to see things line up to the extent that they
177

178
00:16:28,220 --> 00:16:28,490
do.